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@ QAM receiver compensating both linear and non-linear distortion. 

@ The invention relates to an adaptive receiving method and a receiver comprising an adaptive equaling 
device (1) for processing a received signal. The receiver further comprises an adaptive detector (2, 3) utilizing 
the self-organizing map principle and operationally connected after the adaptive equalizer (1). The adaptive 
equalizer (1) is controlled on the basis of an error between a processed signal (y(n)) and a detected signal (y'(n)- 
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The invention relates to a method ot receiving discrete signals, in which a received discrete signal is 
processed prior to detection to compensate for distortion caused by the transmission channel. 

A common problem with the reception of digital signals is that in addition to noise the transmission 
channel also causes linear and non-linear distortions in the signal. Prior art methods for eliminating the 
5 effects of linear distortions from the received signal include linear and non-linear transversal equalizers, 
which may be adaptive so that they adapt to possible changes in the transmission channel during the signal 
transmission. 

To compensate for non-linear distortions, International Application PCT/FI89/00037 discloses an adap- 
tive detection method for quantized signals, which utilizes the self-organizing map principle during detection 
w to automatically take into account the effects of changes taking place in the channel properties on a signal 
constellation used in the detection. The same method is described in Kohonen. Raivio, Simula, VentS 
Henriksson: An Adaptive Discrete-Signal Detector Based on Self-Organizing Maps. International Joint 
Conference on Neural Networks IJCNN-90-WASH DC, Jan. 15-19, 1990, Washington DC. Vol. II, P. 11-249- 
52. 

75 The above-described solutions are not, however, able to operate in an optimal way In surroundings 
where both distortion types occur simultaneously. A linear or non-liner transversal equalizer is not able to 
compensate for non-linear distortions (or is able only limitedly), and the map method compensating for non- 
linear distortions is disturbed by the linear distortions. 

The object of the present invention is to provide a receiving method and a receiver which are able to 

20 efficiently compensate for non-linear and linear distortions simultaneously. 

This is achieved by means of a method of the type disclosed in the introduction, which is according to 
the invention characterized in that the processed discrete signal is detected by an adaptive detection 
method based on a self-organizing map and the distortion compensation to be performed prior to detection 
is controlled on the basis of an error between an instantaneous signal sample of the processed signal and a 

25 signal constellation formed by the self-organizing map. 

The invention eliminates or alleviates the drawbacks of the prior methods by introducing a combined 
method in which the signal is first equalized linearly by a desired adaptive method, e.g., by using a linear 
transversal equalizer or decision-feedback equalizer and the equalized signal is detected by a detection 
method based on the self-organizing map. Thereafter the error of an instantaneous signal sample of the 

30 equalized signal is calculated with respect to a signal constellation formed by the self-organizing map. The 
error term so obtained is used to control the linear equalizer, e.g., the determination of the tap coefficients 
of the equalizer. In one embodiment of the invention, the equalization is made even more efficient by using 
and controlling simultaneously both a conventional linear equalizer and a decision-feedback equalizer. 

As compared with a conventional transversal equalizer, the most significant difference is that the error 

35 term controlling the linear equalizer is calculated in this method with respect to an adaptive signal 
constellation. In traditional methods, the signal constellations and decision limits used in the detection are 
fixed so that they do not take into account changes possibly taking place in the transmission channel due to 
the ageing of the apparatus or for some other reasons. 

In the preferred embodiment of the invention, the signal constellation formed by the self-organizing map 

40 and to be used in the detection is corrected on the basis of the signal the linear distortion of which has 
been equalized. In an alternative embodiment of the invention, the correction of the signal constellation is 
accomplished by the received signal as such, which method in most cases has a lower performance than 
the preferred method but it may provide advantages in cases where non-linear distortion is so extensive that 
the linear equalizer is not able to converge in the initial situation. In a further embodiment of the invention, it 

45 is possible to select adaptively between the two above-mentioned alternatives. 
The invention is also concerned with a receiver according to claim 7. 

The invention will now be described in greater detail by means of embodiments with reference to the 
attached drawings, in which 

Figure 1A illustrates a signal constellation to be used with a 16QAM signal; and 
so Figure 1 B illustrates non-linear distortion of such a signal constellation; 

Figures 2A and 2B illustrate one- and two- dimensional self-organizing maps; 

Figures 3, 4 and 5 are block diagrams of various receivers according to the invention; Figure 6 shows 
one quadrant of the 16QAM signal constellation, illustrating the determination of an error term e(n) used 
in the method of the invention; and 
55 Figures 7 and 8 are block diagrams of two receivers of the invention when the signal to be received is a 
16QAM signal or the like. 

The primary purpose of a typical communication system is to transmit data or messages from one point 
to another. To achieve this, the messages must be coded and modulated in an optimal way to make them 



2 



EP 0 465 851 A2 



as suitable for transmission as possible. Especially long-distance communication usually requires a high- 
frequency carrier, which is modulated by the coded messages. In the well-known amplitude modulation 
(AM), for instance, the carrier is simply multiplied by the message signal. Phase and frequency modulations 
are also used generally. 

5 The transmission channel bridges the two ends of the system. The carrier and the message are usually 
affected by signal attenuation, noise, interference, distortion, etc. A standard method to suppress noise and 
interference is to use proper filtering. For linear distortions, various so-called equalizing techniques have 
been developed. However, many distortions have non-linear characteristics, which make them difficult to 
compensate. In digital communication, modulating signals assume only discrete values at sampled time 

w positions. The problem at the receiver is thus to identify the discrete values, e.g. ±Ac, i3Ac,... One of the 
most efficient modulation techniques is the Quadrature-Amplitude Modulation (QAM). It is based on having 
two identical carriers simultaneously in the same channel with a 90 degrees phase shift. Both carrier 
components can be modulated independently, whereby the signal x(t) is obtained from the equation 

T5 x(t) = x,(t)AcCos(« c t + ehXq(t)AcSin(« c t + e) (1 ) 



The coefficient x,(t)Ac is called the "in-phase" i component and the coefficient Xq(t)Ac the "quadrature" 
q component. In the coordinate system shown in Figure 1 A, the horizontal axis represents the phase of the i 

20 component and the vertical axis represents the phase of the q component so that the phase of the signal x- 
(t) is the sum of the two component phases. In a digital QAM signal, the i and q components can take only 
discrete values. Thus, in the above-mentioned coordinate system, each possible (i,q) pair occupies a 
discrete grid point so that a so-called signal constellation is formed. Demodulation is an inverse operation to 
modulation, i.e. it attempts to recover the discrete X| and Xq codes from the transmitted waveform. More 

25 precisely, the problem is to detect discrete signal values ±Ac , ±3 C , etc. when the signal levels are affected 
by the noise, interference, distortions, etc. Figure 1A also shows the point density function (pdf) of the 
received signals in ideal circumstances, whereby the function contains peaks at the possible grid points. In 
Figure 1B, the peaks of the pdf have become widened due to noise and, moreover, the peaks themselves 
have been shifted due to distortion. Distortion is usually caused by slowly changing phenomena, such as a 

30 change in temperature in the circuitry or transmission medium so that in the short term, let alone noise, the 
discrete signal levels of two successive received signals are not too far apart. Linear distortion can easily be 
compensated for by various equalization techniques but non-linear distortion is more difficult to cope with. 

Simultaneous equalization of linear and non-linear distortion is accomplished efficiently according to the 
basic principles of the invention by equalizing the linear distortion of the received signal and by detecting 

35 the equalized signal by an adaptive detection method based on a self-organizing map. In the detection 
method, the used signal constellation adaptively adapts to the signal states of the received signal distorted 
by the transmission channel. The distortion may take place, e.g., from the ideal signal constellation of 
Figure 1A to the non-linearly distorted signal constellation of Figure 1B. In the invention, the linear equalizer 
is further controlled on the basis of an error between an instantaneous signal sample of a processed signal 

40 y(n) and the signal constellation formed by the self-organizing map. 

Figure 3 is a block diagram of a preferred embodiment of a receiver for discrete signals according to 
the invention for realizing the method. A discrete signal x(n) received from the transmission channel is fed 
to the input of a linear equalizer 1 to reduce or eliminate the linear distortion caused by the channel. An 
output signal y(n) from the equalizer 1, i.e. the processed received signal is fed to a control input in a self- 

45 organizing map 2 and to an input in a detector 3. Adaptive values are developed in the cells of the self- 
organizing map 2, which values follow up changes taking place in time In the discrete signal states of the 
signal y(n) and form an adaptive signal constellation m f (n) which is fed to the detector 3, in which the m, 
values are used as decision levels so that the detector interprets the nearest signal point in the signal 
constellation mrfn) as the value of the instantaneous signal sample of the signal y(n) and gives this value to 

so the detected signal y'(n). The detection error is e(n) = m,(n)-y(n) and it is used for controlling the equalizer 
1 according to the invention. In the preferred embodiment of the invention the processed signal y(n) and the 
detected signal y*(n) are fed to a subtraction circuit 4 which forms the difference signal e(n) of these two 
signals. The difference signal is feed backed to the linear equalizer 1 to adaptively alter the coefficients of 
the linear equalizer in a suitable manner by using, e.g., the gradient method (MMSE criterion) or the zero 

55 compression method. The formation of the error term e(n) is illustrated in Figure 6, which shows one 
quadrant in the signal constellation mrfn) formed by the self-organizing map 2 in the case of a 16QAM 
signal. On account of the linear distortion occurring in the channel, a point m 3 in the signal constellation has 
shifted from its ideal place. The error term e(n) to be used for controlling the linear equalizer is calculated 
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by the instantaneous signal sample y(n) obtained from the output of the linear equalizer 1 and the signal 
point rrt3 used in the detection. 

Figure 4 shows another receiver realizing the method of the invention. The received signal x(n) is fed to 
the inputs of the self-organizing map 2 and the linear equalizer 1. The output signal y(n) of the equalizer 1 

5 is detected by the detector 3, at which the self-organizing map 2 forms the adaptive signal constellation m r 
(n) used in the detection. A difference signal e(n) is formed from the signals y(n) and y'(n) by the 
subtraction circuit 4, which difference signal controls the linear equalizer 1 according to the invention. In this 
embodiment, the self-organizing map forms the signal constellation mi(n) to be used in the detection 
directly on the basis of the received signal x(n). which may be of advantage when the signal x(n) is so 

w severely distorted that the equalizer 1 is not able to converge in the initial situation. 

Figure 5 shows a third receiver realizing the method of the invention. This receiver is formed by adding 
a decision-feedback equalizer (DFE) to the receiver of Figure 3. Accordingly, the blocks indicated with the 
same reference numerals in Figures 3 and 5 represent the same functions or circuits. In Figure 5. the 
decision-feedback equalizer comprises a unit 14 accomplished e.g. by a transversal filter. The detected 

75 signal y'(n) is applied to the input of the unit 14. and It is processed by the unit, whereafter the output signal 
of the unit 14 is subtracted from the output signal of the linear equalizer 1 in a subtraction circuit 15 and the 
obtained difference signal is fed to the detector 3. The error signal e(n) is also fed to the unit 14, and It 
controls the formation of the coefficients of the unit 14 similarly as in the case of the equalizer 1. This 
combination makes the equalization of the distortion even more efficient. 

20 The decision-feedback equalizer can also be similarly positioned in the receiver of Figure 4. It is also 
possible to realize the receiver of the invention merely by the decision-feedback equalizer without the 
equalizer 1 . 

In the following the operation of the different blocks of Figures 3. 4 and 5 will be described in greater 
detail. The equalizer 1 may be any adaptive linear or non-linear equalizer known in the art, which is used to 
25 compensate for channel distortions and intersymbol interference. The most common equalizer structure is 
an adaptive transversal filter, the output signal of which is obtained from the equation 



N 

30 y(n) = Z c(k)x(n-k) (3) 

k=-N 

where the tap coefficients c(n) represent the impulse response of the filter and y(n) is an estimate for the 

as nth input sample. Various suitable equalizer structures are described e.g. in Proakis: Advances in 
Equalization for Intersymbol Interference, Advances in Communication Systems Theory and Applications, 
Vol. 4, Academic Press, 1975. 

The adaptive detection based on the self-organizing map, which is performed by the blocks 2 and 3, is 
described in International Patent Application PCT/FI89/00037 and the following articles: Kohonen, Raivio, 

40 Simula. Venta, Henriksson: An Adaptive Discrete-Signal Detector Based on Self-Organizing Maps. 
International Joint Conference on Neural Networks I J C N N-90- WAS H DC, Jan. 15-19, 1990. Washington DC, 
Vol. II. P. II-249-52; T. Kohonen, Clustering, Taxonomy, and Topological Maps of Patterns, Sixth 
International Conference on Pattern Recognition, Munich, Germany, October 19-22, 1982, p. 114-128; T. 
Kohonen, Self-Organization and Associative Memory, Springer-Verlag, Series in Information Sciences, 

45 Vol. 8, Berlin-Heidelberg-New York-Tokyo, 1984, 2nd ed. 1988. 

These publications are incorporated in the description by reference. The structure and operation of the 
blocks 2 and 3 will, however, be described in short below. A typical self-organizing map is a linear array of 
learning cells, each cell containing an adaptive parameter or signal point mi. The map may be one- 
dimensional (Figure 2A) or two-dimensional (Figure 2B). When the ideal and distorted signal values are one- 

50 dimensional, the m, values are also scalars. In the beginning of the communication, the mi values are 
initialized to the ideal values or according to the signal levels received in the beginning of the transmission, 
or they may be random values because the mi values will effectively converge to possible asymptotic 
values of the received quantized values in the course of the self-organizing learning process. The adaptive 
and time-varying signal identification performed by the blocks 2 and 3 proceeds according to the following 

55 rules that are based on the original self-organizing algorithm. 

(i) At each discrete time instant t, the cell c with the best matching parameter mi(t) for the detection 
result of the current received signal sample x(t) is determined. In the preferred embodiment of the 
invention, the parameter mi to which the Euclidean distance of the signal sample is the smallest is 
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chosen, i.e. 



ltx(t)-m c (t)D = min,{llx(t)-mi(t)0} (4) 

(ii) Adapt the parameters mi in the neighbourhood N c of the chosen cell c 



5 
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m l (t+l)=m i (t)+a[x(t)-m i (1:)] , 
m i (t+l)»m i (t)+6[x(t)-in :L (t:)] / 
m ± ( t+1 )=m i ( t ) 
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The topological neighbourhood N c consists of the selected cell itself and its direct neighbours up to 
75 depth 1, 2,... (see Figure 2B). 

The neighbourhood learning is always applied symmetrically in each direction in the array of adaptive 
cells. Because cells near the edges of the array may not have neighbours in both directions, the learning 
causes some bias in the mi values of these cells towards the parameter in the center of the group. In one 
embodiment of the invention, this is compensated by modifying the input signal x(t) to the form b| + d|X(t), in 
20 which bi and d, are node-specific parameters, i.e. the space of the input signal x(t) is effectively enlarged. 

The ability of the method to preserve a signal space topology becomes even more evident when the 
quantization of the signal space is two- or multi-dimensional, i.e. the ideal signal values occupy the 
coordinate values of the grid points of a rectangular area and the mj values are two-dimensional vectors. 
Otherwise, the above adaptation equations are directly applicable to the two-dimensional case, too. Two- 
25 dimensional signal quantization is utilized in the so-called QAM coding. Therefore it is advantageous to use 
a respective rectangular array of learning cells familiar from the multitude of demonstrations given of the 
self-organizing maps. 

As a result of such adaptation based on self-organizing, the algorithm is able to follow up distortions in 
the signal constellation if the distortions in the receiver are such that the local order of the peaks of the 

30 point density function is preserved. The grid points in a two-dimensional space may be shifted, zoomed, 
rotated, etc., in various ways but still the order of the signal levels tends to be preserved, i.e., the 
rectangular grid-like structure is preserved. 

Figures 7 and 8 show two receivers in which the received signal s(t) may be e.g. an N-phase modulated 
signal or a MQAM signal, such as a 16QAM signal. In Figure 7, the received signal s(t) is detected to the 

35 base frequency by means of mixers 5 and 8 by using a local oscillator 15 and a phase shifter 14 to produce 
local carriers having a phase shift of 90° . The output signals of the mixers 5 and 8 are lowpass-filtered in a 
usual way in filters 6 and 9, respectively. The lowpass-filtered signals are sampled, one sample for each 
transmitted symbol, by switches 10 and 11 which may be e.g. sampling and holding circuits. After 
sampling, signal samples x»(n) and x Q (n) occurring at symbol time slots T are obtained, which are the 

40 quadrature-phase signal components of the QAM signal. 

The signal samples xj(n) and xo(n) are applied to the linear equalizer 1, in which the sample is 
subjected to an equalization depending on the linear equalizer used in each particular case. For example, a 
transversal equalizer comprises signal samples in the memory from the time period of several symbols and 
by suitably emphasizing these signal samples in a manner depending on the tap coefficients and the 

45 impulse response of the equalizer, a corrected signal pair yi(n) and y Q (n) is made to correspond to each 
sampled time position. The samples y,(n) and y Q (n) are digital words of e.g. 8 bits. 

Thereafter the signal samples y ( (n) are applied both to a decision circuit 7 and the self-organizing 
network 2, which together form an adaptive detector. The self-organizing map 2 calculates an optimal signal 
constellation m, for a two-dimensional signal by a method based on self-organizing maps. The most 

so probable values are thereby calculated by means of the detected signal samples y,(n) and y Q (n) for the 
signal points mj, taking into account the noise, interference and distortions contained in the signal. The 
decision circuit 7 utilizes these signal points mj as reference points when it decides to which transmitted 
symbol the received signal sample pair yi(n) and yo(n) corresponds. In the preferred embodiment of the 
invention, the criterion for deciding is that the received signal sample pair corresponds to the reference 

55 point mi to which the Euclidean distance lly-mjll is the smallest The decision circuit 7 also calculates an 
error term e the components of which are 



e, = y,- y'. 
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e Q = y Q - y' Q 

where y'i and y Q are the quadrature-phase output signals of the decision circuit 7. The error term e is used 

as described above for calculating and updating the tap coefficients of the linear equalizer. 
5 The receiver of Figure 8 is otherwise similar to that of Figure 7 except that the linear equalizer is now 

an analog equalizer which is positioned to directly equalize a received signal s(t) before the signal is mixed 

to the base frequency. The signals are not converted into digital form until by analog-to-digital converters 12 

and 13 positioned after the sampling switches 10 and 11. 

Many of the operational blocks shown in Figures 3, 4, 5, 7 and 8 can also be realized programmatically 
w by means of a microprocessor or the like, whereby the self-organizing map 2, the detector 3 and the 

decision circuit 7 shown as separate in the figures may, in fact, be different parts of the programme of one 

and the same processor. 

It is further possible to combine the operations of Figures 3 and 4 in the receiver in such a way that the 
received signal x(n) directly or the signal y(n) processed by the equalizer 1 can be connected adaptively 
75 according to a given decision making criterion as the control signal of the self-organizing map 2. In the 
beginning of the transmission, for instance, the control of the map 2 may be connected directly to the signal 
to be received to facilitate the converging of the linear equalizer 1 and then to the processed signal for the 
rest of the transmission time. 

The features disclosed in the foregoing description, in the following claims and/or in the accompanying 
20 drawings may, both separately and in any combination thereof, be material for realising the invention in 
diverse forms thereof. 

Claims 

25 1. Method of receiving discrete signals, comprising processing a received discrete signal prior to 
detection to compensate for distortion caused by the transmission channel, characterized in that the 
processed discrete signal is detected by an adaptive detection method based on a self-organizing map 
and the distortion compensation to be performed prior to detection is controlled on the basis of an error 
between an instantaneous signal sample of the processed signal and a signal constellation formed by 

30 the self-organizing map. 

2. Receiving method according to claim 1, characterized in that the signal constellation formed by the 
self-organizing map and to be used in the detection is corrected on the basis of the unprocessed, 
received signal. 

35 

3. Receiving method according to claim 1 , characterized in that the signal constellation formed by the 
self-organizing map and to be used in the detection is corrected on the basis of the processed, 
received signal. 

40 4. Receiving method according to claim 1, 2 or 3, characterized in that the self-organizing map and the 
signal constellation formed by it are multi-dimensional. 

5. Receiving method according to any of the preceding claims, characterized in that the calculation of 
the tap coefficients of a linear or non-linear transversal equalizer is controlled by means of said error 

45 using a gradient method or a zero compression method. 

6. Receiving method according to any of the preceding claims, characterized in that a decision-feedback 
equalizer is controlled by means of said error. 

so 7. Receiver comprising an adaptive equalizing device (1) for processing a received signal, characterized 
in that the receiver comprises an adaptive detecting means (2, 3) utilizing the self-organizing map 
principle and operationally connected after the adaptive equalizer (1), and that the adaptive equalizer 
(1) is arranged to be controlled on the basis of an error between a processed signal (y(n)) and a 
detected signal (y'(n)). 

55 

8. Receiver according to claim 7, characterized in that the detecting means (2, 3) is responsive to the 
unprocessed, received signal to correct the signal constellation used in the detection by a method 
based on a self-organizing map. 
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9. Receiver according to claim 7, characterized in that the detecting means (2, 3) is responsive to the 
signal processed by the equalizer device to correct the signaJ constellation used in the detection by a 
method based on a self-organizing map. 

10. Receiver according to claim 7, characterized in that the detecting means is selectively responsive to 
the processed or unprocessed received signal to correct the signal constellation used in the detection 
by a method based on a self-organizing map. 

11. Receiver according to any of claims 7 to 10, characterized in that the received signal is a QAM signal, 
and that the self-organizing map of the detecting means and the signal constellation are multi- 
dimensional. 

12. Receiver according to any of claims 7 to 11, characterized in that the adaptive equalizer comprises a 
linear or non-linear transversal equalizer and/or a decision-feedback equalizer.. 

13. Receiver according to any of claims 7 to 12, characterized in that it comprises a decision-feedback 
equalizer (14, 15). 
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© OAM receiver compensating both linear and non-linear distortion. 



© The invention relates to an adaptive receiving 
method and a receiver comprising an adaptive 
equalizing device (1) for processing a received sig- 
nal. The receiver further comprises an adaptive de- 
tector (2, 3) utilizing the self-organizing map princi- 
ple and operationally connected after the adaptive 
equalizer (1). The adaptive equalizer (1) is controlled 
on the basis of an error between a processed signal 
(y(n)) and a detected signal (y'(n)). 
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